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Session  Outline 


•  What  are  Manufacturing  Readiness  Levels  (MRLs) 

•  Implementing  MRLs  into  a  Program 

•  What  is  a  Manufacturing  Readiness  Assessment  (MRA)? 

•  How  to  perform  a  MRA 

•  Sample  Outputs  and  Deliverables 

•  Findings  and  Conclusions 

•  Additional  Information 


What  are  MRLs? 


•  Common  language  and  standard  for 

-  Assessing  the  manufacturing  maturity  of  a  technology  or  product 
and  plans  for  its  future  maturation 

-  Understanding  the  level  of  manufacturing  risk  in  trying  to  produce  a 
weapon  system  or  transition  technology  into  a  weapon  system 
application 

•  Designed  to  complement  TRLs 

•  Designed  to  help  set  the  agenda  for 
manufacturing  risk  mitigation 

•  Establish  an  expectation  of  achieving 
manufacturing  maturity  at  critical  decision 
points 

NOTE:  FY04  -  Joint  Defense  Manufacturing  Technology 
Panel  (JDMTP)  formed  MRL  Working  Group  to 
develop  basic  MRL  definitions  and  criteria 


What  are  TRLs? 


•  Provide  a  common  language  and  widely- 
understood  standard  for: 

•  Assessing  the  performance  maturity  of  a 
technology  and  plans  for  its  future  maturation 

•  Understanding  the  level  of  performance  risk  in 
trying  to  transition  the  technology  into  a  weapon 
system  application 


System  Test.  Launch 
&  Operations 


System/Subsystem 

Development 


Technology 

Demonstration 


Technology 

Development 


TRL9 


TRL8 

TRL7 


TRL4 


Research  to  Prove 
Feasibility 


TRL3 
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Why  MRLs? 


•Acquisition  Performance  perceived  as  broken 

•Numerous  Studies  conducted  in  last  25  Years  to  improve  Acquisition 
Process 

•Packard  Commission,  Goldwater -Nichols  Act,  SecDef  Perry  Memo, 
etc 


•GAO  continues  to  identify  cost  and  schedule  overruns  on  Major  DoD 
Acquisition  Programs 

•Need  better  knowledge  early  in  acquisition  to  guide  decisions 

•Congress  trying  to  help  improve  Acquisition  Process 
•TRL  Legislation 

•Weapons  Systems  Acquisition  Reform  Act  of  2009 

•New  DoDI  5000.02  adding  new  requirements  to  perform  manufacturing 
assessments 

•BOTTOM  LINE  Current  Performance  in  Acquisition  must  be  improved  - 
MRLs  just  one  key  tool  to  help 
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New  Emphasis  on  Manufacturing 


Per  5000.02,  all  programs  shall: 

assess  the  manufacturing  feasibility  and  risk  of  alternatives  being  evaluated  in  the 
Analysis  of  Alternatives, 

assess  the  manufacturing  feasibility  and  risk  of  the  proposed  materiel  solution  for 
use  in  the  Milestone  A  decision, 

conduct  assessments  to  identify  manufacturing  risks  and  ensure  manufacturing 
processes  have  been  demonstrated  in  a  relevant  environment  in  conjunction  with 
the  Preliminary  Design  Review  (PDR)  for  use  in  the  Milestone  B  decision, 

conduct  assessments  to  ensure  the  maturity  of  critical  manufacturing  processes 
has  been  demonstrated  in  a  representative  environment  in  conjunction  with  the 
Critical  Design  Review  (CDR),  and 


conduct  assessments  to  ensure  that  manufacturing  processes  have  been 
effectively  demonstrated  in  a  pilot  line  environment  and  that  there  are  no 
significant  manufacturing  risks  for  use  in  the  Milestone  C  or  FRP  decisions. 
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How  does  MRLs  Help 


•Great  Management  Tool 


•Defines  Disciplined  Process  -  Industry  Best  Practice 
•Analogous  to  Stage  Gates  for  new  product 
development 


•Improves  Communications  and  alignment  of  purpose 
between  key  stakeholders: 

•Government  and  Contractor 
•Prime  and  Suppliers 

•Industry’s  functional  organizations  (i.e.  engineering, 
manufacturing,  supply  chain,  quality,  and  program 
office 


•Opportunity  to  generate  fact-based  technical  discussions 


MRLs  A  Great  Management  Tool 


•Structured  Approach 
•Develops  Knowledge 

-  Manufacturing  processes,  characteristics,  constraints, 
requirements 

Supports  analysis  of  cost,  schedule,  producibility,  scale-up, 
trades 

•Identifies  &  Manages  Risk 

-  Key  manufacturing  risks  identified  as  early  as  milestone  A 

-  Comprehensive  assessment  of  total  manufacturing 
requirements 

•Great  Program  Review  Tool 

-Assess  manufacturing  progress  at  reviews 

•Products/Processes  now  aligned  prior  to  entering  production 

•Can  be  used  both  tactically  to  solve  a  problem  and  strategically  to 
identify  Global  and  Enterprise  issues  that  need  to  be  addressed 
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MRLs  A  Best  Practice 


•Good  Commercial  Companies  address  manufacturing 
maturity  early  in  development 

•Provides  discipline  in  addressing  key  manufacturing 
areas 

•Prime  and  Suppliers  on  same  work  sheet 
•Requires  both  Government  and  Industry  to  address 
manufacturing  risk 

•  Developed  with  SMEs  from  over  60  major  DoD 
suppliers  and  Multiple  Government  Agencies 
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MRLs  A  Great  Communication 

Tool 


•Standardize  methodology  across  multiple 
suppliers 

•  Requirements  to  achieve  a  defined  level  of 
manufacturing  maturity  is  consistent 

•  Provides  consistency  across  the  industrial 
base 

•Provides  a  consistent  set  of  expectations 
throughout  the  life  cycle 

•Identifies  systemic  problems  across  programs, 
suppliers,  and  services 
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Key  Variables  for  Successful  MRL 

Implementation 


•Program  leadership  commitment 

•Doing  MRL  Assessments  early  in  Acquisition 
Process  can  lead  to  greater  impact 

•Using  qualified  SMEs  to  effectively  implement 
and  assess 


•Follow  through  on  planning  and  execution  of 
necessary  maturity/risk  reduction  efforts 
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MRL  Benefits 


•MRLs  provide  a  lot  of  value  at  very  little  cost 
•Benefits  significantly  out  weigh  cost 

•Cost  to  implement  and  manage  MRLs  should  not  be  considered  “added 
cost”  but  rather  early  focus  of  risk  mitigation  efforts  that  tend  to  occur 
later,  after  the  fact,  and  in  worst  case  scenarios.... not  at  all 


•MRLs  provide  the  required  early  visibility  into  manufacturing  maturity  to 
effectively  manage  risk 

•MRLs  set  measurable  and  performance-based  goals  and  provide  oversight 
of  contractor’s  performance  at  both  Prime  and  Suppliers 

•MRLs  provide  clear  expectations  on  what’s  expected  -more  effective 
planning  from  everyone 

•MRLs  provide  needed  visibility  and  encourages  proactive  risk  management 

•Implementing  MRL  process  can  answer  the  new  DoDI  5000.02 
requirements  for  performing  manufacturing  assessments 

12 


Today’s  Air  Force  Reality 


•  Diminishing  manufacturing  infrastructure 

-  People,  policy,  programs  gutted 

-  Lost  recipe  on  how  to  manage  manufacturing  risk 

-  Won’t  get  infrastructure  back,  but  still  need  to  manage 
and  mitigate  manufacturing  risk 
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MRL  &  TRL  Relationships 
In  Acquisition  Process 


Relationship  to  System  Acquisition  Milestones 


Materiel  Solution  Analysis 

Technology 

Development 

Engineering  & 
Manufacturing 
Development 

Production  & 
Deployment 

A 

L  A 

i  A 

MRL  1 

Basic  Mfg 
Implications 
Identified 


MRL  2 

Mfg 

Concepts 

Identified 


MRL  3 
Mfg 

Proof  of 
Concept 
Developed 


MRL  4 

Manufacturing 
Processes 
In  Lab 

Environment 


MRL  5 

Components 
In  Production 
Relevant 
Environment 


MRL  6 

System  or 
Subsystem 
In  Production 
Relevant 
Environment 


MRL  7 

System  or 
Subsystem 
In  Production 
Representative 
Environment 


MRL  8 

Pilot  Line 
Demonstrated 
Ready  for 
LRIP 


MRL  9 

LRIP 

Demonstrated 
Ready  for 
FRP 


MRL  10 

FRP 

Demonstrated 
.ean  Production 
Practices  in 
place 


TRL  1 

TRL  2 

TRL  3 

TRL  4 

TRL  5 

TRL  6 

TRL  7 

TRL  9 

TRL  9 

Basic 

Concept 

Proof 

Breadboard 

Breadboard 

Prototype 

Prototype 

Mission 

Mission 

Principles 

Observed 

Formulation 

of 

Concept 

in 

Lab 

in  Rep 

Environment 

in  Rep 

Environment 

in  Ops 

Environment 

Proven 

Proven 

Relationship  to  Technology  Readiness  Levels 
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MRL  Definitions 


an u facto ring  Readiness  Level  Definitions 


Version  7.0-  October.  2008 


MRL 

Definition 

Description 

Phase 

1 

“Esse 

Manufacturing 

Imp  bca  tons 
Identrfied. 

rase  '===3':'-  3ctv:  esexca-a  sce-rrc  c"ic  ces:-a:  may  '•avemanjiacc'ng 

impfccatons  The  focus  ts  on  a  high  level  assessment  of  manufactinng  opportunfies  The 
■•esearch  ts  unfettered. 

-re  Concept 

Refinement 

2 

Manufacturing 

Concepts 

Identified. 

invention  ceg  ns  Manufacturing  sc  ence  ana;  or  concept  s  deserned  n  app.  cation 

context.  Identficason  of  mater  a  l  and  process  approaches  are  m  ted  to  paper  studies 
and  analysis  ln<d  manufacturing  feasibly  and  issues  are  emerging 

-re  concept 

Refinement 

3 

Manufacturing 

Proof  of  Concept 
Developed. 

Analytical  or  3cor3tory  ex:er  ments  a-e  conducted  to  va  cate  pace'  stud  es 

Experimental  hardware  or  processes  have  been  created,  but  are  not  yet  integrated  or 
representative.  Materials and.1  or  processes  havebeen  characteraed  for  manufacturaWky 
and  avaiabflky  but  further  evaluation  and  demonstrason  ts  required 

-re  concept 

Refinement 

4 

Capability  to 

produce  the 
technology  m  a 
laboratory 
environment. 

Required  invest  me**  ts  .such  as  manufactu^-g  technology  deveopmen:  den:  red 

=rocesses  to  ensure  manufacturability1,  produciWky  and  quality  are  m  place  and  a'e 
s  efficient  to  produce  technology  demonstrators.  Manufacturing  ns  *cs  identified  for 
prototype  buM.  Manulactunng  cost  drivers  identified.  Productoitty  assessments  of 
design  concepts  have  ceen  completed  Key  Performance ^aramete-s  (KPP)  identified. 
Spec  s!  needs  identified  for  tooling,  facilities .  material  handling  and  skills. 

concept 

Refinement 
(CR)  leading  to 
a  Milestone  A 
decision. 

5 

Capabiftyto 

produce 

prototype 

components  m  s 

production 

relevant 

environment. 

Mtg  st'ategy  -et  nedand  integrate]  wijiksc  *-sn  idenr  -at on  ot  e^ao  ng.cr.tca 

technologies  and  components  s  complete.  Prototype  matenals.  toahng  and  test 
equipment,  as  wel  as  personnel  stasis  have  been  demonstrated  on  componerts  m  a 
production  relevant  environment,  but  many  manufacturing  processes  and  procedures  are 
stfl  in  development  Manufacturing  technology  development  efforts  masted  or  ongoing 
Frodueforfov  assessments  of  key  technologies  and  components  ongoing  Cost  model 
cased  upon  detailed  end-to-end  value  stream  map. 

lecnno.ogy 

Development 
(YD)  Phase 

6 

Capabiftyto 

produce  a 
prototype  system 
or  subsystem  m  a 
production 
relevant 
environment. 

in  its  mtg  approacr  deveoced.  M3  prty  of  manjtactj'rg  excesses  nave  teen  det  -ed 

and  ctia'acterzed.  but  there  are  stfl  significant  eng  neerng.des  gn  changes  Prefcmmary 
desgn  of  critical  components  completed.  Productoibty  assess  ments  of  key  technooges 
complete.  Prototype  materids.toofing  and  testequ  pmen.  as  wel  as  personnel  skfls 
have  Ceen  demonstrated  on  subsystems.1  systems  n  a  production  relevant  environment, 
□eta  ed  cost  analyze  includes  des  gn  trades  Costtargetsa  located  ProduciWky 
considerations  shape  system  development  plans.  Long  lead  and  key  supply  chan 
elements  identified  Industrie  CapaWifees  Assessment  (CA)for  MS  B  completed 

leennoogy 

Development 
(TD)  phase 
ead  -ng  to  a 

decision 

7 

C909Dilrtyto 

produce  systems 
subsystems  or 
components  in  9 
production 
representative 
environment. 

jets  e:  cesgr  a  urce^ay.  Mate^a  spec:  tstK-rs  3'=  approve:  Mate's  sava  abefo 
meet  planned  pilot  line  build  schedule.  Manufacturing  processes  ard  procedures 
demonstrated  in  a  production  represertatrve  environment  Detailed  producibiiity  trade 
studies  and  risk  assessments  underway.  Cost  models  updated  with  detailed  designs, 
rolled  up  to  system  level  and  tracked  againsttargefs.  Unit  cost  reduction  efforts 
underway.  Supply  chain  and  supplier  CA  assessed.  Long  lead  procurement  plans  in 
place.  Production  tooling  and  test  equipment  des  ign  &  development  initiated. 

Development 
&  Demo(SDD) 
leading  to 
Desgn 

R  e3d  ires  s 
Review  (DRR> 

8 

Pilot  line 

capability 
demonstrated. 
Re9dyto  begin 
low  rate 
production. 

production.  All  materials  are  available  to  meet  planned  low  rate  production  schedule. 
Manufacturing  and  quality  processes  and  procedures  proven  in  a  pilot  line  environment, 
under  control  and  ready  for  low  rate  production.  Known  producibiiity  risks  pose  no 
significant  risk  for  low  rate  production.  Engineering  cost  model  driven  by  detailed  design 
and  validated.  Supply  chair  established  arc  stable.  ICAfor  MS  C  completed. 

system 
Development 
&  Demo 
leading  to  a 
Milestone  C 
decision. 

9 

Low  Rate 

Production 

demonstrated. 

Ca  pa  bity  in 
place  to  begm 

Ful  Rale 
Production. 

Map' system  desgn  features aro  st3be and  proven  m  test andevaiuafor  Malena s 3re 

available  to  meet  planned  rate  production  schedules  Manufacturing  processes  and 
procedures  are  established  and  contrcfled  to  three-sgma  or  some  other  appropriate 
quality  level  to  meet  desgn  key  characteristic  toleranoes  m  a  low  rate  production 
environment  Production  nsk monuonng  ongoing. LRIP cost goas  met.  Seam .ng  curve 
validated  Actual  cost  model  developed  for  FRP  environment  with  impact  of  Continuous 
improvement 

-rooucfio^^ 

Deployment 
lead  mg  to  a 

Ful  Rate 
Production 
(FRP)  dec  sen 

10 

Ful  R9te 

Production 
demonstrated 
and  lean 
production 
practices  in 
place. 

ins  stnengnest  level  or  production  read  ness,  fcngmccnnyoes  gn  cnanges  are  Tew 

and  generally  kmked  to  quahty  and  cost  improvements  System .  components  or  kerns  are 
m  rate  production  and  meet  all  engineering.  performance,  quality  and  reiabiity 
requirements  All  matenals.  manufacturing  processes  and  proced’jes.  nspection  and 
test  equipment  are  «  production  and  controlled  to  soc-sgma  or  some  other  appropriate 
a. a  j:::;  aaa  ::=  ;.=  ir:1:  :  a:,  a:  a  ^ 

_ean  practices  wel  established  and  continuous  process  improvements  ongoing 

hun  Kate 

Production,' 

Sustamment 

MRL  Definitions  con’t 


MRL  1 

MRL  2 

MRL  3 

MRL  4 

MRL  5 

MRL  6 

MRL  7 

MRL  8 

MRL  9 

MRL  10 

Basic  mfg 

Mfg 

Mfg  proof 

Capability  to 

Capability  to 

Capability  to 

Capability  to 

Pilot  line 

Low  rate 

Full  rate 

implications 

concepts 

of  concept 

produce  the 

produce 

produce  a 

produce 

capability 

production 

production 

identified 

identified 

developed 

technology  in 

prototype 

prototype 

systems, 

demonstrated. 

demonstrated. 

demonstrated 

a  laboratory 

components 

system  or 

subsystems  or 

Ready  to 

Capability  in 

pnrl  Ippn 

environment 

in  a 

subsystem  in 

components 

begin  low  rate 

place  to  begin 

production 

production 

a  production 

in  a 

production 

full  rate 

relevant 

relevant 

production 

production 

practices  in 

environment 

environment 

representative 

place 

environment 

- 1 

L _ 

_ A _ 

- 1 

L — 

•  Production  relevant  environment -An  environment  with  some  shop  floor  production  realism  present  (such  as  facilities,  personnel, 
tooling,  processes,  materials  etc.).  There  should  be  minimum  reliance  on  laboratory  resources  during  this  phase.  Demonstration 
in  a  production  relevant  environment  implies  that  contractor(s)  must  demonstrate  their  ability  to  meet  the  cost,  schedule,  and 
performance  requirements  of  the  EMD  Phase  based  on  their  production  of  prototypes.  The  demonstration  must  provide  the 
program  with  confidence  that  these  targets  will  be  achieved.  Furthermore,  there  must  be  an  indication  of  how  the  contractor(s) 
intend  to  achieve  the  requirements  in  a  production  representative  and  pilot  environments. 

•  Production  representative  environment -An  environment  that  has  as  much  production  realism  as  possible,  considering  the 
maturity  of  the  design.  Production  personnel,  equipment,  processes,  and  materials  that  will  be  present  on  the  pilot  line  should  be 
used  whenever  possible.  The  work  instructions  and  tooling  should  be  of  high  quality,  and  the  only  changes  anticipated  on  these 
items  are  associated  with  design  changes  downstream  that  address  performance  or  production  rate  issues.  There  should  be  no 
reliance  on  a  laboratory  environment  or  personnel. 

»  Pilot  line  environment -An  environment  that  incorporates  all  of  the  key  production  realism  elements  (equipment,  personnel  skill 
levels,  facilities,  materials,  components,  work  instructions,  processes,  tooling,  temperature,  cleanliness,  lighting  etc.)  required  to 
manufacture  production  configuration  items,  subsystems  or  systems  that  meet  design  requirements  in  low  rate  production.  To  the 
maximum  extent  practical,  the  pilot  line  should  utilize  full  rate  production  processes. 
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9  MRL  Evaluation  Criteria 

(“Threads”) 


1.  Technology  and  Industrial  Base 

•  Technology  maturity,  technology  transition  to  production,  ManTech  development 


2.  Design 

•  Producibility  program,  design  maturity 

3.  Cost  and  Funding 

•  Production  cost  knowledge  (cost  modeling),  cost  analysis,  mfg  investment  budget 

4.  Materials  (raw  matls,  components,  subassys,  subsystems) 

•  Maturity,  availability,  supply  chain  management,  special  handling 

5.  Process  Capability  and  Control 

•  Modeling  &  Simulation  (product  &  process),  mfg  process  maturity,  process  yields/rates 

6.  Quality  Management,  to  include  supplier  quality 

7.  Manufacturing  Personnel,  to  include  specialization,  training,  & 
certification 

8.  Facilities,  to  include  capacity  and  plant  layout  &  design 

9.  Manufacturing  Management 

•  Manufacturing  planning  and  scheduling 

•  Materials  planning 

•  Tooling  and  special  test  equipment 


MRL  Criteria  Matrix 


DoD  Manufacturing  Readiness  Levels  (MRLs) 


Acq  Phase 

Pre  CR 

Pre  CR 

Pre  CR 

CR  -  MS  A— 

k.  TD 

MS  B  — ► 

^  SDD 

MS  C  ^ 

^  LRIP  -  FRP  ^ 

^  FRP 

Thread 

Sub-Thread 

MRL  1 

MRL  2 

MRL  3 

MRL  4 

MRL  5 

MRL  6 

MRL  7 

MRL  8 

MRL  9 

MRL  10 

Technology  &  Industrial  Base 

Technology 

Maturity 

Should  be  assessed  at  TRL 1. 

Should  be  assessed  at  TRL  2. 

Should  be  assessed  at  TRL  3. 

Should  be  assessed  at  TRL  4. 

Should  be  assessed  at  TRL  5. 

Should  be  assessed  at  TRL  6. 

Should  be  assessed  at  TRL  7 

Should  be  assessed  at  TRL  7. 

Should  be  assessed  at  TRL  8. 

Should  be  assessed  at  TRL  9. 

Technology 
Transition  to 

Production 

Potential  sources  identified  for 
technology  needs.  (Understand 
state  of  the  art). 

Industrial  Base  capabilities  and 
gaps/risks  identified  for  key 
technologies,  components, 
andfor  key  processes. 

Industrial  Base  assessed  to 
identify  potential  manufacturing 

sources. 

Industrial  Capability  Assessment 
(ICA) for  MSB  has  been 
completed.  Industrial  capability  in 
place  to  support  mfg  of 
development  articles.  Plans  to 
minimize  solefforeign  sources 
complete.  Need  for  solefforeign 
sources  justified.  Potential 
alternative  sources  identified. 

Industrial  capability  to  support 
production  has  been  analyzed. 
Solefforeign  sources  stability  is 
assessedfmonitored.  Developing 
potential  alternate  sources  as 
necessary. 

Industrial  Capability  Assessment 
(ICA)  for  MS  C  has  been 
completed.  Industrial  capability  is 
in  place  to  support  LRIP.  Sources 
are  available,  multi-sourcing  where 
cost-effective  or  necessary  to 
mitigate  risk. 

Industrial  capability  is  in  place  to 
support  start  of  FRP. 

Industrial  capability  supports  FRP. 
Industrial  capability  assessed  to 
support  mods,  upgrades,  surge 
and  other  potential  manufacturing 
requirements. 

Manufacturing 

Technology 

Development 

New  manufacturing  concepts  and 
potential  solutions  identified. 

Manufacturing  technology 
concepts  identified  through 
experiments  >  models. 

Mfg  science  5 1  advanced  mfg 
technology  requirements 
identified. 

Required  manufacturing 
technology  development  efforts 
initiated. 

Manufacturing  technology  efforts 
continuing.  Required 
manufacturing  technology 
development  solutions 
demonstrated  in  a  production 

Manufacturing  technology  efforts 
continuing.  Required 
manufacturing  technology 
development  solutions 
demonstrated  in  a  production 

Manufacturing  technology  efforts 
continuing.  Required 
manufacturing  technology 
solutions  validated  on  a  pilot  line. 

Manufacturing  technology  efforts 
continuing.  Manufacturing 
technology  process  improvement 
efforts  initiated  for  FRP. 

Manufacturing  technology  efforts 
continuing.  Manufacturing 
technology  continuous  process 
improvements  ongoing. 

Design 

Producibility 

Program 

Relevant  malerialsfprocesses 
evaluated  for  manufacturability 
using  experimental  results. 

Producibility  &  Manufacturability 
assessment  of  design  concepts 
completed.  Results  guide 
selection  of  design  concepts  and 
key  componentsftechnologies  for 
T echnology  Development 

Strategy.  Required  Test 

Evaluation  Strategy  (TES) 
includes  Design  for  Test  during 
production. 

Producibility  &  Manufacturability 
assessments  of  key  technologies 
and  components  initiated. 

Systems  Engineering  Plan  (SEP) 
requires  validation  of  design 
choices  against  manufacturing 
process  and  industrial  base 
capability  constraints. 
Manufacturing  processes 
assessed  for  capability  to  test  and 
verify  in  production,  and  influence 
on  Operations  &  Support  (O&S). 

Producibility  assessments  of  key 
technologiesfcomponents  and 
producibility  trade  studies 
(performance  vs.  producibility) 
completed.  Results  used  to 
shape  System  Development 
Strategy  and  plans  for  SDD  or 
technology  insertion  programs 
phase. 

Detailed  producibility  trade  studies 
using  knowledge  of  key  design 
characteristics  and  related 
manufacturing  process  capability 
completed.  Producibility 
enhancement  efforts  (e.g.  Design 
For  Mfg  Assembly)  initiated. 

Producibility  improvements 
implemented  on  system.  Known 
producibility  issues  have  been 
resolved  and  pose  no  significant 
risk  for  LRIP. 

Prior  producibility  improvements 
analyzed  for  effectiveness  during 
LRIP.  Producibility  issuesfrisks 
discovered  in  LRIP  have  been 
mitigated  and  pose  no  significant 
risk  for  FRP. 

On-going  producibility 
improvements  analyzed  for 
effectiveness.  Producibility 
refinements  continue.  All  mods, 
upgrades,  Diminishing  Mfg 

Sources  &  Material  Shortages 
(DMSMS)  and  other  changes 
assessed  for  producibility. 

Design  Maturity 

Manufacturing  opportunities 
identified. 

Applications  defined.  Broad 
performance  goals  identified  that 
may  drive  manufacturing  options. 

T op  level  performance 
requirements  defined.  Trade-offs 
in  design  options  assessed  based 
on  experiments.  Product  lifecycle 
and  technical  requirements 
evaluated. 

Initial  Systems  Engineering  Plan 
and  Test  and  Evaluation  Strategy 
recognize  the  need  for  the 
establishmentfvalidalion  of 
manufacturing  capability  and 
management  of  manufacturing 
risk  for  the  product  lifecycle.  Initial 
Key  Performance  Parameters 
(KPPs)  identified. 

Identification  of  enabling/critical 
technologies  and  components  is 
complete  and  includes  the  product 
lifecycle.  Evaluation  of  design  Key 
Characteristics  (KC)  initiated. 

Basic  system  design  requirements 
defined.  All  enabling/critical 
technologiesfcomponents  have 
been  tested  and  validated. 

Product  data  required  for 
prototype  manufacturing  released. 
A  preliminary  performance  as  well 
as  focused  logistics  specification 
is  in  place.  Key  Characteristics 
and  tolerances  have  been 

established. 

Product  requirements  and 
features  are  well  enough  defined 
to  support  detailed  systems 
design.  All  product  data  essential 
for  manufacturing  of  component 
design  demonstration  released. 
Potential  KC  risk  issues  have 
been  identified  and  mitigation  plan 
is  in  place.  Design  change  traffic 
may  be  significant. 

Detailed  design  of  product 
features  and  interfaces  is 
complete.  All  product  data 
essential  for  system 
manufacturing  released.  Major 
product  design  features  are 
sufficiently  stable  such  that  key 
LRIP  manufacturing  processes 
will  be  representative  of  those 
used  in  FRP.  Design  change 
traffic  does  not  significantly 
impact  LRIP.  Key  characteristics 
are  stable  and  have  been 

demonstrated  in  SDD  or 
technology  insertion  program. 

Major  product  design  features  are 
stable  and  LRIP  produced  items 
are  proven  in  product  testing. 
Design  change  traffic  is  limited  to 
minor  configuration  changes.  All 
KCs  are  controlled  in  production 
to  3-sigma  or  other  appropriate 
quality  levels. 

Product  design  is  stable.  Design 
changes  are  few  and  generally 
limited  to  those  required  for 
continuous  improvement  or  in 
reaction  to  obsolescence.  All 

KCs  are  controlled  to  6-sigma  or 
other  appropriate  quality  levels. 

Production  Cost 
Knowledge  (Cost 
modeling) 

Cost  model  approach  defined. 

T echnology  cost  models 
developed  for  new  process  steps 
and  materials  based  on 
experiments. 

Detailed  process  chart  cost 
models  driven  by  key 
characteristics  and  process 
variables.  Manufacturing, 
material  and  specialized  reqt.  cost 
drivers  identified. 

Detailed  end-to-end  value  stream 
map  cost  model  for  major  system 
components  includes  materials, 
labor,  equipment.  loolingfSTE. 
setup,  yieldfscrapfrework,  Work  In 
Progress  (WIP),  and 
capability/capacity  constraints. 
Component  simulations  drive 

Cost  model  inputs  include  design 
requirements,  material 
specifications,  tolerances, 
integrated  master  schedule, 
results  of  systemfsubsystem 
simulations  and  production 
relevant  demonstrations. 

Cost  models  updated  with  detailed 
designs  and  features,  collected 
quality  data,  plant  layouts  and 
designs,  and  obsolescence 
solutions. 

Engineering  cost  model  driven  by 
detailed  design  and  validated  with 
data  from  relevant  environment. 

Actual  cost  model  developed  for 
FRP  environment.  Variability 
experiments  conducted  to  show 
FRP  impact  and  potential  for 
continuous  improvement. 

Cost  model  validated  against 
actual  FRP  cost. 
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Session  Outline 


•  What  are  Manufacturing  Readiness  Levels  (MRLs) 

•  Implementing  MRLs  into  a  Program 

•  What  is  a  Manufacturing  Readiness  Assessment  (MRA)? 

•  Why  Manufacturing  Readiness? 

•  Sample  Outputs  and  Deliverables 

•  Findings  and  Conclusions 

•  Additional  Information 
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MRLs  in  Program 


•  Address  MRLs  in  Key  Documents 

•  SEP,  TDS,  Acq  Plans,  RFP 

•  Use  manufacturing  maturity/risk  in  Source  Selection 

•  Manufacturing  Risk  needs  to  be  integrated  into  Program’s  Risk 
Management  activity 

•  Implementing  MRLs  into  a  Program  Management  Activity 

•  Incorporate  in  IMP  and  IMS 

•  EVS  data 

•  Address  Manufacturing  Maturity  into  Program  Management 
Reviews 

•  Assess  manufacturing  maturity  in  Technical  Reviews 

•  PDRs,  CDRs,  etc 

•  Perform  manufacturing  readiness  assessments 

•  Status  manufacturing  maturity  at  milestone  reviews 

•  Address  DoDI  5000.02  requirements 

•  BOTTOM  LINE  -  Manufacturing  is  important  and  needs  to  be 

addressed  in  your  programs  20 
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•  How  to  perform  a  MRA 

•  Sample  Outputs  and  Deliverables 

•  Findings  and  Conclusions 

•  Additional  Information 


21 


What  is  an  MRA? 


An  evaluation  of  a  program’s  ability  to 
produce  on  schedule  while  meeting  cost  and 
performance  requirements 

A  tool  to  develop  and  implement: 

-  Manufacturing  risk  mitigation  plans 

-  Business  strategies 

Results  in  an  assignment  of  MRLs  to  key 
system  components 


•  Results  in  the  development  of  a 
manufacturing  maturation  plan  as  required 
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MRA  Process 


•  Assessment  Lead  works  with  PM  and  contractor  on 
Manufacturing  Assessments  Process 

-  MRA  Deskbook  spells  out  process 

•  Assessment  lead  responsibilities 

•  Determine  appropriate  level  to  perform  Manufacturing  Readiness 
Assessment(s)  (MRAs)  --  System  may  contain  several  critical 
technologies/components/manufacturing  cells 

•  Schedule  on-site  MRA  with  contractor(s) 

•  Send  Orientation  Package  to  contractor(s) 

•  Define  Assessment  Team  Membership 

•  Define  Deliverables  of  Assessment  Results 

•  Conduct  on-site  assessment  with  contractor(s) 

•  Deliver  final  report/briefing 
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Manufacturing  Readiness  Implementation 

Approach  (ACATs) 


\r 


Preparations 


•  Determine  where  to  perform  MRAs 

-  Address  contract  issues 

•  Notify  companies  and  send  orientation  package 

-  Purpose,  approach,  questions,  strawman  agenda 

-  MRL  definitions/threads 

-  Self-Assessment 

•  Select  Assessment  team(s) 

-  Typically  2-6  members  per  team 

-  Appropriate  members  (include  Gov’t  customer) 

•  Specialists  for  key  technologies  (if  needed) 

•  Schedule  On-site  assessments 

-  Months  prior  to  key  milestone  decisions 

•  Allow  time  to  develop/implement  risk  mitigation  plans 

•  Team  Orientation-Meet  prior  to  on-site  assessment 
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Where  to  perform  MRA 


•  Key  questions  to  ask! 

•  Materials:  Are  there  materials  which  have  not  been  demonstrated  in  similar  products 
or  manufacturing  processes? 

•  Cost:  Is  this  item  a  driver  that  significantly  impacts  life-cycle  cost  (development,  unit, 
or  operations  and  support  costs)?  Is  the  technology  new  with  high  cost  uncertainty? 

•  Design:  Is  the  item  design  novel  or  does  it  contain  nonstandard  dimensions  or 
tolerances  or  arrangements? 

•  Manufacturing  Process:  Will  the  item  require  the  use  of  manufacturing  technology, 
processes,  inspection,  or  capabilities  that  are  unproven  in  the  current  environment? 

•  Quality:  Does  the  item  have  historical/anticipated  yield  or  quality  issues? 

•  Schedule:  Does  this  item  have  lead  time  issues  or  does  it  significantly  impact 
schedule? 

•  Facilities:  Does  this  item  require  a  new  manufacturing  facility  or  scale  up  of  existing 
facilities  (i.e. ,  new  capability  or  capacity)? 

•  Supply  Chain  Management:  Does  the  item  have  anticipated  or  historical  sub-tier 
supplier  problems  (e.g.,  cost,  quality,  delivery)? 

•  Industrial  Base:  Does  the  item  have  an  industrial  base  footprint  with  critical 
shortfalls  or  is  this  a  critical  item  manufactured  by  a  sole  or  foreign  source? 

•  If  yes,  consider  performing  an  MRA  at  this  facility. 
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On-Site  Activities 


•  Introduce  team  and  highlight  specific  roles 

•  Purpose  of  assessment  and  how  results  will  be  used 

•  Assessment  process 

•  Expectations 


•  Contractor  should: 

•  Introduce  contractor  participants  -  highlight  responsibilities 

•  Provide  detailed  responses  to  questions  provided  in  orientation  package 

•  Indicate  who  is  available  to  work  with  team  counterparts  in  later  meetings  for  detailed 
discussions 

•  Lead  shop  floor  tour 


•  Perform  initial  assessment 


•  Private  meeting  of  Government  assessment  team  to: 

•  Prepare  feedback  and  identify  any  action  items 

•  Key  strengths/risks/issues 

•  Key  missing  data  (if  any) 

•  Proposed  action  items 

•  Out-briefing  by  Government  team  to  contractor 
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Example  Process  Flow 
Generic  Aircraft 


•  Large  programs  can  require  multiple  MRAs 


Deliver  to  Facility  B 
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Supplier  MRA  Plan 


•  Identify  and  prioritize  critical  suppliers  to  perform  MRA 

•  Determine  Contract  requirements  (e.g.  SOW) 

-  Scope  of  MRA  detailed 

-  Method  of  MRA  detailed 

-  Output  defined 

•  Developed  detailed  MRA  execution  plan  with  each 
supplier  (schedule,  format,  personnel) 

•  Execute  MRA 

•  Define/Plan/Execute  MRL  mitigations 

•  Measure  mitigation  effectiveness,  update  assessment 

•  NOTE:  Similar  to  prime  MRA  -  Prime  should  lead 
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Follow-on  Activities 


•  Gather  any  key  missing  data 

•  Convene  team  meeting  --  Typically  within  2  weeks  of  on¬ 
site  assessment 

•  Discuss  and  finalize  assessment 

•  Examine  current  program  and  manufacturing  risk  reduction  plans 

•  Agree  on  likely  MRL  at  completion  of  milestone  if  current  plan  is 
followed 

•  Share  results  with  contractor 

•  Identify  the  specific  risk  reduction  activities  necessary  to 
reach  the  next  milestone 

•  Identify  the  funding,  time-phasing  and  approach  to 
carrying  out  each  activity 

•  Prepare  and  submit  final  report 
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MRA  Deliverables 


•  Provide  briefing  and/or  written  report 

•  Identify  target  MRL 

•  Identify  current  MRL 

•  Identify  key  factors  where  manufacturing  readiness  falls 
short  of  target  MRL 

•  Define  driving  issues 

•  Identify  programs  and  plans  to  reach  target  MRL 

•  Assess  type  and  significance  of  risk  to  cost,  schedule  or 
performance 


•  Next  step:  Implement  and  execute  the 
Manufacturing  Maturation  Plan 
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SAMPLE  SUMMARY  ROLL-UP 
OF  COMPONENTS 


Subsystem 

Top  Level 
MRL 

Observations 

Most  Critical 

Guidance 

3 

-  Lacking  detailed  process  information 

-  Key  suppliers  identified;  Need  key 
performance  parameters 

-  Need  detailed  process  plans 

Detector  from  supplier  A 

-  Design  &  production  issues 

-  No  alternate  source 

Data 

Processor 

3 

-  New  processor  architecture 
-Immature  design  tools 

-  New  attachment  processes  needed 

Board  Supplier  can’t  test  at  their 
site 

Low  yields  on  initial  run 

Propulsion 

6 

-  Same  as  other  systems  in  use 

-  New  component  scheme 

Re-validate  manufacturing 
process 

Supplier  handle  increased  rate 

Air 

Vehicle 

7 

-  Same  supplier  as  system  X 

-  Need  to  test  new  mating  and  assembly 
processes  at  the  prime 

No  critical  items 

Test  Plan 

6 

Several  instances  of  re-design  work  and 
new  test  processes 

-  New  test  strategy  and  plan 

-  What  will  new  design 
incorporate? 

-  Manufacturing  experience  vital 
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SAMPLE  SUMMARY 
(Drill  down) 


Guidance 

Sub 

systems 

Top  Level 
MRL 

Observations 

Most  Critical 

Front  End 
Sensor 

3 

-  Lacking  details  on  builds 

-  Process  procedures  need  more  work 

-  Test  and  assembly  procedures  have  not  been 
verified  in  manufacturing  environment 

Detector  from  supplier  A 

-  Design  &  production  issues 

-  No  alternate  source 

Data 

Processing 

PWB 

3 

-  New  processor  architecture 
-Awaiting  Design  for  Manufacturing  and 
Assembly  (DFMA)  results 

Low  yields  on  initial  build 

Working  process  controls 

Looking  at  re-design  for  easier 
fabrication 

Cables  For: 
Power 

Data 

3 

-  Using  same  suppliers  other  weapon  systems 

-  Have  not  received  prototypes,  awaiting 
supplier  delivery 

Re-validate  manufacturing  process 
as  seen  on  past  programs 

Need  new  process  plan 

Housing 

4 

-  New  supplier:  limited  experience 

-  Need  new  assembly  processes  at  the  prime 

Need  supplier  management 
process;  need  new  process  plans 

Cooling 

3 

-  Form,  fit  factors  for  new  cooling  design  not 
in  place 

Initial  process  plan  for  build  in  place 

Final  cooling  plan  will  be  defined 
after  front  end  is  stable 

Integration 
Process  that 
includes 
assembly 
and  test 

3 

-  Several  new  test  processes  need 
development  for  new  components 

New  test  strategy  and  plan 

New  special  test  equipment  must 
be  ordered 
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MRA  Risk  Management 


•  Assessing  Risk  is  independent  of  the  MRL  value  assigned 

-  Higher  MRL  value  may  be  highest  risk 

•  Eg.  Requires  new  equipment,  high  cost  process 

•  Risk  Assessment  should  consider 

-  Time  needed  to  reach  target  MRL 

-  Require  new  personnel,  training,  capital,  or  more  POM  samples  to 
flush  out  the  process 

-  Leverage  other  programs 

-  Captive  or  Merchant  Supplier  Dependency?? 

-  Part  of  a  company’s  core  business 

•  Leads  into  an  industrial  base  assessment 

•  Effective  of  use  of  Design  for  Manufacturing  Tools  and  other 
simulation  techniques. 
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Some  MRL  Thoughts 


•  MRLs  are  not  a  report  card 

-  MRL  7  might  not  be  good 

-  MRL  3  might  not  be  bad 

•  MRLs  are  a  tool  to  manage  and  mitigate 
manufacturing  risk 

-  A  common  language  used  to  assess  manufacturing 
maturity 

-  Provide  insight  not  oversight 
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MRA  Thoughts 


•  MRA  process  highlights  areas  needing  attention 
to  lower  production  risk 

•  Sets  expectation  and  measures  Prime  contractor 
and  Suppliers  Manufacturing’s  progress  as 
program  matures 

•  Detailed  analysis  rank  ordered;  Can  be  an 
investment  strategy  and  feeds  into  risk 
management 

•  Effective  tool  to  convince  Senior  Leaders  you 
are  ready  for  production 


Manufacturing  maturity  through  the  MRA  process 
enables  efficient,  cost  effective  manufacturing 
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Additional  Information 


•  DoD  Manufacturing  Readiness  Reference 

http://www.dodmrl.com/ 

•  MRL  Definitions 

•  MRL  Criteria  Matrix 

•  MRL  Deskbook 

•  Defense  Acquisition  University  MRL  Website 

•  AF  ManTech’s  MRATool  and  Users  Manual 
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